Naive-Bayes for Sentiment Classification

This report details the findings in building a naive Bayes sentiment classifier for a IMDB movie-review data set using Scala and ScalaNLP. We studied the unigram or bagof-words Bernoulli and Multinomial models and a number of different feature selection techniques, including term frequency, mutual information and Chi-squared. 1. DATA CORPUS The corpus contains of 2000 rated movie reviews, comprised of an equal number of reviews for each sentiment group (positive and negative). Without stemming or stop-words removal there are a total of 1293948 word and 48813 unique words in the dictionary. On average, each document in the positive sentiment corpus contains 683 words, 350 of which are unique, while each document in the negative sentiment corpus contains 610 words, 325 of which are unique. Given the relatively lengths of documents, we expect a-priori that a multinomial model to produce more accurate results than the Bernoulli model. Note that the data was partitioned by sentiment into positiveand negative‘folders’ a-priori, so no explicit tagging was required. 2. MODELS Unigram, or a bag-of-words multinomial and Bernoulli models were considered. Bigrams will be explored as future work. Naive-Bayes (NB) makes the fundamental assumption that the words in a document are conditionally independent given the class. Thus, the joint-likelihood of observing a sequence of terms conditioned on the class c easily decouples: P (t1, t2, . . . , tn|c) = ∏